-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Shared: Provenance-based filtering of flow summaries #21051
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Shared: Provenance-based filtering of flow summaries #21051
Conversation
a3e585d to
eb48820
Compare
1e946f8 to
30a0791
Compare
0fbea88 to
5a2881d
Compare
5a2881d to
a941f4a
Compare
bf632b3 to
c6383ff
Compare
9f81377 to
0057ae3
Compare
1933d1c to
72dfe9c
Compare
72dfe9c to
5642898
Compare
| } | ||
|
|
||
| class URLConstructor extends DataFlow::SummarizedCallable { | ||
| class URLConstructor extends DataFlow::SummarizedCallable::Range { |
Check warning
Code scanning / CodeQL
Acronyms should be PascalCase/camelCase
| * callable, (b) there is no manual (neutral) model, and (c) the model is inexact | ||
| * and there is no generated exact (neutral) model. | ||
| */ | ||
| final class RelevantSummarizedCallable extends SummarizedCallableFinal { |
Check warning
Code scanning / CodeQL
Suggest using non-extending subtype relationships
2d63aaa to
4060c02
Compare
Click to show differences in coveragejavaGenerated file changes for java
- `Apache Commons Collections <https://commons.apache.org/proper/commons-collections/>`_,"``org.apache.commons.collections``, ``org.apache.commons.collections4``",,1600,,,,,,,
+ `Apache Commons Collections <https://commons.apache.org/proper/commons-collections/>`_,"``org.apache.commons.collections``, ``org.apache.commons.collections4``",,1615,,,,,,,
- Java Standard Library,``java.*``,10,4628,260,99,,9,,,26
+ Java Standard Library,``java.*``,10,4629,260,99,,9,,,26
- `Spring <https://spring.io/>`_,``org.springframework.*``,46,492,143,26,,28,14,,35
+ `Spring <https://spring.io/>`_,``org.springframework.*``,46,494,143,26,,28,14,,35
- Totals,,363,26372,2681,404,16,134,33,1,409
+ Totals,,363,26390,2681,404,16,134,33,1,409
- java.util,48,2,1339,,,,,,,,,1,,,,,,,,,,,34,,,,3,,,,5,2,,1,2,,,,,,,,,,,,,,2,,,558,781
+ java.util,48,2,1340,,,,,,,,,1,,,,,,,,,,,34,,,,3,,,,5,2,,1,2,,,,,,,,,,,,,,2,,,558,782
- org.apache.commons.collections4,,,800,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,17,783
+ org.apache.commons.collections4,,,815,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,17,798
- org.springframework.web.util,,9,157,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9,132,25
+ org.springframework.web.util,,9,159,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9,134,25 |
Missing manual models were added using the following code added to `FlowSummaryImpl.qll`:
```ql
private predicate testsummaryElement(
Input::SummarizedCallableBase c, string namespace, string type, boolean subtypes, string name,
string signature, string ext, string originalInput, string originalOutput, string kind,
string provenance, string model, boolean isExact
) {
exists(string input, string output, Callable baseCallable |
summaryModel(namespace, type, subtypes, name, signature, ext, originalInput, originalOutput,
kind, provenance, model) and
baseCallable = interpretElement(namespace, type, subtypes, name, signature, ext, isExact) and
(
c.asCallable() = baseCallable and input = originalInput and output = originalOutput
or
correspondingKotlinParameterDefaultsArgSpec(baseCallable, c.asCallable(), originalInput,
input) and
correspondingKotlinParameterDefaultsArgSpec(baseCallable, c.asCallable(), originalOutput,
output)
)
)
}
private predicate testsummaryElement2(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string originalInput, string originalOutput, string kind, string provenance, string model
) {
exists(Input::SummarizedCallableBase c |
testsummaryElement(c, _, _, _, _, _, _, originalInput, originalOutput, kind, provenance,
model, false) and
testsummaryElement(c, namespace, type, subtypes, name, signature, ext, _, _, _, provenance,
_, true) and
not testsummaryElement(c, _, _, _, _, _, _, originalInput, originalOutput, kind, provenance,
_, true)
)
}
private string getAMissingManualModel() {
exists(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string originalInput, string originalOutput, string kind, string provenance, string model
|
testsummaryElement2(namespace, type, subtypes, name, signature, ext, originalInput,
originalOutput, kind, provenance, model) and
result =
"- [\"" + namespace + "\", \"" + type + "\", True, \"" + name + "\", \"" + signature +
"\", \"\", \"" + originalInput + "\", \"" + originalOutput + "\", \"" + kind + "\", \"" +
provenance + "\"]"
)
}
```
4060c02 to
b6764b2
Compare
This PR aligns the logic across languages for how flow summaries are prioritized based on provenance and exactness (that is, whether a model is defined directly for a function or for a function that is implemented/overridden).
A flow summary is considered relevant if:
Note that for dynamic languages we currently pretend that no source code is available for functions with flow summaries, so 3.(a) holds vacuously.
Point 2 represents a change for e.g. Java, where we would previously union exact and inexact manual models, which meant that it was not possible to overrule inexact models. As a consequence, some inexact models now have to be replicated.
In order for the logic to be defined in the shared flow summary library, I had to move provenance and exactness information into the
propagatesFlowpredicate, which is a breaking change.Lastly, I have applied the
::Rangepattern to theSummarizedCallableclass for all languages except C++, which currently does not expose this class. This means thatSummarizedCallable::Rangewill contain all flow summaries, whereasSummarizedCallablewill only contain relevant summaries.